Skip to main content

Folder

Adding Folder as a data source

  1. From the left navigation panel, click Lakehouse and then click Data Sources.

  2. From the upper right corner of the page, click the + New Database button to start the process of adding a new database.

  3. In the New Data Source page, click the Folder icon.

  4. Specify the following details to add folder. Once you have connected a data source, the system immediately fetches its schema. After this schema retrieval process is complete you can browse and interact with the tables and data.

    Add folder
    FieldDescription
    Connection nameEnter a unique name for the connection.
    File FormatSpecify any of the following file formats: CSV, Parquet, JSON, Excel
    SeparatorSpecify the separator character.
    HeaderToggle to indicate if the first row of your CSV contains column headers. This field is displayed only when the file format is CSV.
    Infer SchemaToggle to automatically determine the data type of each column in your data. This field is displayed only when the file format is CSV.
    CompressionSelect the file compression mode from any of the following options: Uncompressed, gzip, lzo, brotli, lz4, and zstd.
    Null ValueA set of case-sensitive strings that should be interpreted as null values. For example, if the value 'NA' should be interpreted as null, enter 'NA' in this field.
  5. Click Submit.

After creating a data source, add a new folder to define a unique data table. Any files uploaded to this folder will be automatically merged to populate the table's data.

Define-unique-data-table

To upload, download, or delete files within a specific folder, click the folder name or access the settings menu. The Files tab will provide you with the necessary options.

Define-unique-data-table1
NOTE
  • Adding files of different schemas in the same folder is not supported.

  • Multi tab excel files are not supported.

  • Folder names cannot be only numbers. Validation is yet to be added.